doc: create ai-guidelines and include to CONTRIBUTING#62105
doc: create ai-guidelines and include to CONTRIBUTING#62105RafaelGSS wants to merge 4 commits intonodejs:mainfrom
Conversation
Co-Authored-By: Beth Griggs <bethanyngriggs@gmail.com>
|
Review requested:
|
|
There may be some ideas we can borrow from https://llvm.org/docs/AIToolPolicy.html - for example "good first issue" should not be picked up by AI is a good one. |
Co-authored-by: Aditi <62544124+Aditi-1400@users.noreply.github.com> Co-authored-by: Joyee Cheung <joyeec9h3@gmail.com>
I took inspiration from https://github.com/zulip/zulip/blob/main/CONTRIBUTING.md#ai-use-policy-and-guidelines |
| * **Verify accuracy** of any LLM-generated content before including it in a | ||
| PR description or comment. | ||
| * **Complete pull request templates fully** rather than replacing them with | ||
| LLM-generated summaries. |
There was a problem hiding this comment.
Do we have a template? I thought those are for issues, not PRs.
There was a problem hiding this comment.
Not strictly a template: https://github.com/nodejs/node/blob/main/.github/PULL_REQUEST_TEMPLATE.md?plain=1
There was a problem hiding this comment.
It's not possible to fulfil the instructions "Complete pull request templates fully" based on the contents of https://github.com/nodejs/node/blob/main/.github/PULL_REQUEST_TEMPLATE.md?plain=1 so it looks like this sentence needs to be removed.
Co-authored-by: Tobias Nießen <tniessen@tnie.de> Co-authored-by: Antoine du Hamel <duhamelantoine1995@gmail.com> Co-authored-by: Mike McCready <66998419+MikeMcC399@users.noreply.github.com> Co-authored-by: Efe <dogukankrskl@gmail.com>
There was a problem hiding this comment.
I'd be against this contribution policy update. While many different opinions exist on the licensing terms of the code produced by LLMs, my opinion is that the generated code isn't explicitly licensed and attributed to the original authors so it cannot be considered open source regardless of the used prompt.
| * **Verify accuracy** of any LLM-generated content before including it in a | ||
| PR description or comment. | ||
| * **Complete pull request templates fully** rather than replacing them with | ||
| LLM-generated summaries. |
There was a problem hiding this comment.
| * **Verify accuracy** of any LLM-generated content before including it in a | |
| PR description or comment. | |
| * **Complete pull request templates fully** rather than replacing them with | |
| LLM-generated summaries. | |
| * **Verify accuracy** of any LLM-generated content before including it in a | |
| PR description or comment. |
|
In #61478 (comment) , regarding the usage of Claude Code, @mcollina suggested:
I added it to the TSC agenda tomorrow for awareness/context collection before moving to a proper vote. @indutny sorry about the short notice since this is just one day ahead of the meeting, but if you'd like to join the meeting to present your points please let us know. |
mcollina
left a comment
There was a problem hiding this comment.
lgtm with a sentence removed
|
@joyeecheung thanks for considering me for this! I'd be happy to join if the time isn't in conflict with my work meetings tomorrow. Could you send me an invite, please? |
|
nodejs/TSC#1830 this is the issue. It might be reschedueld to the next meeting if there is low attedance given the timezone. I think we can schedule the discussion for April 1st which will be in the morning PT so a lot of people can join, and I would table the vote for that session. I would try to get an answer from the Board by then. |
|
Let's make sure it's on the agenda for a time more convenient for the majority of TSC members. That said, I can say right now that I'm -1 on blocking this on those grounds. |
|
I've updated the agenda tomorrow to mark it FYI (as in, it doesn't need to be discussed this week, but worth taking a look and do some homework before discussions happen). In any case, decisions won't be made in meetings, and this likely needs a proper vote. |
| * **Own every line you submit.** You are responsible for all code in your | ||
| pull request, regardless of how it was generated. Be prepared to explain | ||
| any change in detail during review. |
There was a problem hiding this comment.
| * **Own every line you submit.** You are responsible for all code in your | |
| pull request, regardless of how it was generated. Be prepared to explain | |
| any change in detail during review. | |
| * **Own every line you submit.** You are responsible for all code in your | |
| pull request, regardless of how it was generated. This includes ensuring | |
| that AI-generated or AI-assisted contributions satisfy the project's | |
| [Developer's Certificate of Origin][] and licensing requirements. Be | |
| prepared to explain any change in detail during review. |
(would need a DCO link added at the bottom of the page also)
|
[from vote on AI and in context of quoted there PR] If someone uses an advanced tool, like LLM, should everyone have a higher baseline for expectations? Take a look at initial review in github.com/#61478 , looks like begging to use node's style for ... core node. This tool, LLM, is supposed to do things in style. Instead there was a re-implementation of path.normalize() ? User of an advanced tool has responsibility to provide more advanced result. Of course, in coding, LoC count is not a metric for being better. For my 25 years, usage of more advanced tooling brought better results. Should LLM's code be considered a starting point, not a ready PR? May I note, as user of node.js, an unhealthy dynamic regarding this, and a need to frame constructive approach, also as an example for whole industry. nodejs/TSC#1831 (comment) is opened with, quote: "In order to surpass the blocks of ..." as if we are here to score social points, using heavy words. Please, snap out of it. We need to re-frame topic in nuances that matter to code. Otherwise, we either get more hack-able artifacts, or, throw away baby with the bath water. And you all here seem to have enough emotional shrapnel to swing either way. Please don't. For the sake of us, your users. |
|
|
||
| ## Core principle | ||
|
|
||
| Node.js expects contributors to understand and take full responsibility for |
There was a problem hiding this comment.
| Node.js expects contributors to understand and take full responsibility for | |
| Node.js requires contributors to understand and take full responsibility for |
| ## Core principle | ||
|
|
||
| Node.js expects contributors to understand and take full responsibility for | ||
| every change they propose. The answer to "Why is X an improvement?" should |
There was a problem hiding this comment.
| every change they propose. The answer to "Why is X an improvement?" should | |
| every change they propose. The answer to "Why is X an improvement?" can |
| AI tools may assist contributors, but must not replace contributor judgment. | ||
| When using AI as a coding assistant: |
There was a problem hiding this comment.
| AI tools may assist contributors, but must not replace contributor judgment. | |
| When using AI as a coding assistant: | |
| Contributors may use AI tools to assist with contributions, but such tools | |
| never replace contributor judgement. | |
| When using AI as a coding assistant: |
| Node.js values concise, precise communication that respects collaborator time. | ||
|
|
||
| * **Do not post messages generated entirely by AI** in pull requests, issues, or the | ||
| project's communication channels. |
There was a problem hiding this comment.
I'm less convinced this one is necessary. It's also difficult to enforce. These should follow the same rules as contributions... whatever is posted, you're responsible for, so use appropriate discretion.
| contributor has not personally understood, tested, and verified might be closed | ||
| without review. |
There was a problem hiding this comment.
| contributor has not personally understood, tested, and verified might be closed | |
| without review. | |
| contributor has not personally understood, tested, and verified will likely be closed | |
| without review. |
|
|
||
| ## Using AI for code contributions | ||
|
|
||
| AI tools may assist contributors, but must not replace contributor judgment. |
There was a problem hiding this comment.
| AI tools may assist contributors, but must not replace contributor judgment. | |
| AI tools may assist contributors, but must not replace human judgment. |
| * **Understand the codebase first.** Do not skip familiarizing yourself with | ||
| the relevant subsystem. LLMs frequently produce inaccurate descriptions of | ||
| Node.js internals — always verify against the actual source. When using an AI | ||
| tool, ask it to cite the exact source files/PRs/docs it’s relying on, and then |
There was a problem hiding this comment.
| tool, ask it to cite the exact source files/PRs/docs it’s relying on, and then | |
| tool, ask it to cite the exact source it’s relying on, and then |
|
|
||
| ## Using AI for communication | ||
|
|
||
| Node.js values concise, precise communication that respects collaborator time. |
There was a problem hiding this comment.
| Node.js values concise, precise communication that respects collaborator time. | |
| Node.js values concise, precise communication that respects collaborator and contributor time. |
| * **Link to primary sources** — code, documentation, specifications — rather | ||
| than quoting LLM answers. |
There was a problem hiding this comment.
| * **Link to primary sources** — code, documentation, specifications — rather | |
| than quoting LLM answers. | |
| * **Link to primary sources** — code, documentation, specifications — rather | |
| than quoting LLM answers or linking to LLM chats. |
As discussed in today's TSC meeting.
cc: @nodejs/tsc @BridgeAR